Columns removed from the table appear in the Column Chooser box until they're dragged back into the table.